---
title: "Add New Connector"
edit-this-page-url: https://github.com/airweave-ai/airweave/blob/main/fern/docs/pages/add-new-source.mdx
slug: add-new-source
---

Connectors are how Airweave pulls data from external systems, turns it into entities, and makes it searchable for agents. If your favorite tool or API is not yet supported, you can build a connector and either use it locally or contribute it back to the community.

This guide will walk you through the full process step by step:

- Setting up your environment and forking the repo
- Defining authentication and configuration schemas
- Creating entity definitions for your data
- Implementing the source connector itself
- Testing your connector inside the Airweave dashboard

Once complete, your connector will behave like any built-in integration. You can create collections, sync data, and query it through Airweave’s search APIs or MCP server.

If you plan to contribute your connector back to the project, please also review our [Contributing Guidelines](https://github.com/airweave-ai/airweave/blob/main/CONTRIBUTING.md)
 for details on branch naming, development workflow, and pull request process. For help at any point, you can join the [Airweave Discord](https://discord.com/invite/484HY9Ehxt)
 to connect with other contributors and maintainers.

 ## How to add a Connector

<Steps>

  <Step title="Fork the repository" toc={true}>
    Get the [code](https://github.com/airweave-ai/airweave/fork) and set up your development environment.

    ```bash
    git clone https://github.com/YOUR_USERNAME/airweave.git
    cd airweave
    ./start.sh
    ```

  </Step>

  <Step title="Create authentication schema" toc={true}>
       Define what credentials your source needs in [`backend/airweave/platform/configs/auth.py`](https://github.com/airweave-ai/airweave/blob/main/backend/airweave/platform/configs/auth.py).

        ```python
        class GhibliAuthConfig(AuthConfig):
            """Studio Ghibli authentication credentials schema."""

            # No authentication needed for this public API
            pass
        ```
  </Step>

  <Step title="Create source configuration" toc={true}>
    Define optional configuration options in [`backend/airweave/platform/configs/config.py`](https://github.com/airweave-ai/airweave/blob/main/backend/airweave/platform/configs/config.py).

        ```python
        class GhibliConfig(SourceConfig):
            """Studio Ghibli configuration schema."""

            include_rt_scores: bool = Field(
                default=True,
                title="Include RT Scores",
                description="Whether to include Rotten Tomatoes scores in the metadata"
            )
        ```
  </Step>

  <Step title="Define entity schemas" toc={true}>
     Create entity schemas in [`backend/airweave/platform/entities/your_source.py`](https://github.com/airweave-ai/airweave/tree/main/backend/airweave/platform/entities) that define the structure of the data in the source.

        ```python maxLines=100
        from datetime import datetime
        from typing import List, Optional

        from pydantic import Field

        from airweave.platform.entities._base import ChunkEntity

        class GhibliFilmEntity(ChunkEntity):
            """Schema for a Studio Ghibli film."""

            film_id: str = Field(..., description="Unique ID of the film")
            title: str = Field(..., description="Title of the film")
            original_title: str = Field(..., description="Original Japanese title")
            director: str = Field(..., description="Director of the film")
            producer: str = Field(..., description="Producer of the film")
            release_date: str = Field(..., description="Release date")
            running_time: str = Field(..., description="Running time in minutes")
            rt_score: Optional[str] = Field(None, description="Rotten Tomatoes score")
            people: List[str] = Field(default_factory=list, description="Characters in the film")
            species: List[str] = Field(default_factory=list, description="Species in the film")
            locations: List[str] = Field(default_factory=list, description="Locations in the film")
            vehicles: List[str] = Field(default_factory=list, description="Vehicles in the film")
        ```

    **Key points about entities:**
        - Inherit from `ChunkEntity` for searchable content (documents, posts, issues)
        - Inherit from `FileEntity` for downloadable files (PDFs, images, attachments)
        - Use `Field(...)` for required fields, `Field(default=...)` for optional ones
        - Add source-specific fields that are relevant for search and metadata

  </Step>

  <Step title="Implement source" toc={true}>
    Create your source connector in [`backend/airweave/platform/sources/your_source.py`](https://github.com/airweave-ai/airweave/tree/main/backend/airweave/platform/sources).

        ```python maxLines=100
        from typing import AsyncGenerator, Optional, Dict, Any
        import httpx

        from airweave.platform.entities.ghibli import GhibliFilmEntity
        from airweave.platform.decorators import source
        from airweave.platform.sources._base import BaseSource
        from airweave.platform.auth.schemas import AuthType

        @source(
            name="Studio Ghibli",
            short_name="ghibli",
            auth_type=AuthType.none,
            auth_config_class="GhibliAuthConfig",
            config_class="GhibliConfig",
            labels=["Entertainment", "API"]
        )
        class GhibliSource(BaseSource):
            """Studio Ghibli source implementation."""

            @classmethod
            async def create(
                cls,
                credentials=None,
                config: Optional[Dict[str, Any]] = None
            ) -> "GhibliSource":
                """Create a new Ghibli source instance."""
                instance = cls()
                instance.config = config or {}
                return instance

            async def generate_entities(self) -> AsyncGenerator[GhibliFilmEntity, None]:
                """Generate entities from the Ghibli API."""
                async with httpx.AsyncClient() as client:
                    response = await client.get("https://ghibli.rest/films")
                    response.raise_for_status()
                    films = response.json()

                    for film in films:
                        yield GhibliFilmEntity(
                            entity_id=film["id"],
                            film_id=film["id"],
                            title=film["title"],
                            original_title=film["original_title"],
                            content=film["description"],  # Required by ChunkEntity
                            director=film["director"],
                            producer=film["producer"],
                            release_date=film["release_date"],
                            running_time=film["running_time"],
                            rt_score=film["rt_score"] if self.config.get("include_rt_scores", True) else None,
                            people=film.get("people", []),
                            species=film.get("species", []),
                            locations=film.get("locations", []),
                            vehicles=film.get("vehicles", [])
                        )
        ```

    **Key implementation points:**
        - Import your custom entity classes
        -  Use the `@source()` decorator with your auth and config classes
        - Implement `create()` classmethod that handles credentials and config
        - Implement `generate_entities()` that yields your custom entity objects
        - Handle authentication based on your auth type
        - Use config options to customize behavior
  </Step>

  <Step title="Test your source" toc={true}>
    Verify everything works by running Airweave and creating a test connection.

    1. **Start Airweave**: Your connector appears automatically in the dashboard
    2. **Create a collection** and add your new source
    3. **Test the connection** and verify data syncs correctly
    4. **Search your data** to confirm everything works end-to-end
  </Step>

</Steps>

### File Structure

Your complete implementation will create files in these locations:

<CodeBlock title="File structure">
```
backend/airweave/platform/
├── configs/
│   ├── auth.py              # Add YourSourceAuthConfig class
│   └── config.py            # Add YourSourceConfig class
├── entities/
│   └── your_source.py       # Define entity schemas for your data
└── sources/
    └── your_source.py       # Your source implementation
```
</CodeBlock>

### Notes

Be careful about what kind of authentication your app uses. Airweave supports many auth types including API keys, various OAuth2 flows, and database connections. Check your data source's API documentation to determine the correct `AuthType` to use in your `@source()` decorator.

For OAuth2 sources, you'll also need to add your integration to the [`dev.integrations.yaml`](https://github.com/airweave-ai/airweave/blob/main/backend/airweave/platform/auth/yaml/dev.integrations.yaml) file.

### Contributing

<CardGroup cols={2}>
  <Card
    title="Contribution Guidelines"
    icon="fa-solid fa-code-pull-request"
    href="https://github.com/airweave-ai/airweave/blob/main/CONTRIBUTING.md"
  >
    Follow our guidelines for contributing new connectors
  </Card>
  <Card
    title="Get Help"
    icon="fa-brands fa-discord"
    href="https://discord.gg/484HY9Ehxt"
  >
    Join our Discord for help building your connector
  </Card>
</CardGroup>
